QTM 385 - Experimental Methods
Lecture 06 - Texts for Discussion and More
Danilo Freire
Emory University
Hi, there! 👋
Tudo bem? 😄🇧🇷
Last time, we saw that…
- Regression in R: Covered adding covariates with
lm_lin(), centring, and sub-group analysis with interaction terms using lm_robust.
- Hypothesis Testing: Covered different types of null hypotheses: sharp null and null hypothesis of no average treatment effect; confidence intervals, type I and type II errors
- Statistical concept of sampling variability, how it relates to sampling distributions, and how it affects the interpretation of experimental estimates
- Randomisation Inference: Introduced, with the
ri2 package, which uses the randomizr package.
- Demonstrated RI with two examples, including one about female council heads and the other about runoff elections.
- Real Experiment: Analysed a paper using linear regression models as well as Randomisation Inference
Today, we will…
- Talk a bit about
R and the packages we have seen so far
- Discuss the next steps of your pre-analysis plan and the final project
- Discuss three papers
- Kalla, J. & D. Broockman. 2016. Campaign Contributions Facilitate Access to Congressional Officials: A Randomized Field Experiment. American Journal of Political Science, 60(3): 545–558
- Chattopadhyay, R., & Duflo, E. (2004). Women as policy makers: Evidence from a randomized policy experiment in India. Econometrica, 72(5), 1409-1443.
- Shrout, P. E., & Rodgers, J. L. (2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69(1), 487-510.
- But first…, let’s talk about your group work
Group Work 🤝
What I have in mind…
- Now that you have your groups, I’d like you to start working on your pre-analysis plan
- I’ll give you some time to discuss your ideas and start writing your plan
- My plan is that you should slowly think about your final project, one step at a time
- What do you think about having two weeks to write the following:
- Submit at most 2 paragraphs summarising an experiment that you want to develop in this course. At minimum, your summary should include a research question, why the question is important, and a rough sketch of how you plan to answer the question.
- In three weeks:
- Write a title and abstract for a paper you imagine writing based on your proposed experiment. Assume that your findings align with your theoretical predictions. Remember to establish why the findings matter for your intended audience.
- In four weeks:
- Outline your pre-analysis plan. Your outline should include sections on the research question, the experimental design, the data you will collect, and the analysis you will conduct.
What I have in mind…
- In five weeks:
- Use
Quarto and DeclareDesign to write your report and simulate your experiment
- In six weeks:
- Revised outline, now including a new section titled “Potential Threats.” In this section, diagnose threats and briefly describe potential countermeasures. This new section should discuss false positives, statistical power, demand effects, noncompliance, spillover, and attrition.
- In seven weeks:
- Revised outline, now included a new section on “Heterogeneous Treatment Effects.” In this section, discuss how you might explore heterogeneity in treatment effects. This section should include a discussion of how you might use covariates to explore heterogeneity.
- In eight weeks:
- I will give you feedback on your outline and provide you with the simulated data for your experiment.
- You will have two weeks to write your final report.
- Last week of class:
- You will present your findings to the class.
fabricatr
fabricatr is a package that helps you simulate data for your experiments